-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor and simplify logic in sdksCommon cmake script #3251
Conversation
70619ca
to
612cabe
Compare
get_map_element(${PROJECT_NAME} TEMP_VAR ${SDK_DEPENDENCY_LIST}) | ||
get_flatmap_value(${PROJECT_NAME} TEMP_VAR ${SDK_DEPENDENCY_LIST}) | ||
if (TEMP_VAR) | ||
string(REPLACE "," ";" TEMP_LIST_VAR ${TEMP_VAR}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
one thing I see is because get_flatmap_value returns a cmake list, it already separates by semicolon, so we do not need this line I think. It doesn't hurt but will simplify even more on removal
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I saw this and even tried to get rid of this replacements as well, but
while it is indeed tries to return a semicolon-separated cmake list,
thanks to some cmake deep "magic", it is returned as a comma-separated list, and then it starts to fail in other places of the project in very strange ways (like when both comma and semicolon just disappear and the whole list is concatenated into a single word).
We can address it later (if it is even possible to fix)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting. Thanks for letting me know
LIST(APPEND C2J_SPECIAL_NAME_LIST "dynamodbstreams:streams.dynamodb") | ||
set (svc "dynamodbstreams") | ||
elseif("${svc}" MATCHES "\\.") | ||
string(REPLACE "." ";" nameParts ${svc}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These special name mapping is also captured in https://github.com/aws/aws-sdk-cpp/blob/main/tools/code-generation/smithy/codegen/smithy2c2j_service_map.json
Please just ensure the original naming is retained otherwise smoke tests wont build because they are built using the cmake library name which mimics the folder name
example: https://github.com/aws/aws-sdk-cpp/blob/main/generated/smoke-tests/apigateway/CMakeLists.txt#L4
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For reference, here is how the cmake is generated:
Line 27 in 94627af
"aws-cpp-sdk-" + folderNamespace + "\n" + |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please read the PR description first, I've actually documented this:
- c2j file renaming logic is extracted and duplicated within the code generation script run_code_generation.py, so additional logic of datetime extraction, c2j filename renaming and etc is not actually required in the cmake script.
I've run an additional test today, just in case:
# Build main branch and install
# git branch: main
rm -rf buildMainClean && cmake -B buildMainClean -S . -DCMAKE_INSTALL_PREFIX=/home/srryabin/sdk-install-main -DENABLE_SMOKE_TESTS=ON -DCMAKE_BUILD_TYPE=Debug -DCMAKE_TOOLCHAIN_FILE=toolchains/clang-c++20.cmake &> configureMain.log && cmake --build buildMainClean -j 32 --config Debug --target install &> buildMain.log
# Build this branch and install
# git branch: sr/cmakeListClients
rm -rf buildRefactor && cmake -B buildRefactor -S . -DCMAKE_INSTALL_PREFIX=/home/srryabin/sdk-install-refactor -DENABLE_SMOKE_TESTS=ON -DCMAKE_BUILD_TYPE=Debug -DCMAKE_TOOLCHAIN_FILE=toolchains/clang-c++20.cmake &> configureRefactor.log && cmake --build buildRefactor -j 32 --config Debug --target install &> buildRefactor.log
# Compare the build output, the only difference is the output from the test testing race conditions on the logging (which is pretty okay)
tree buildRefactor > cmakerefac_tree.log
tree buildMainClean > cmakeclean_tree.log
diff cmakeclean_tree.log cmakerefac_tree.log
1c1
< buildMainClean
---
> buildRefactor
24739a24740
> │ ├── aws_sdk_2025-01-21-18.log
24847c24848,24849
< │ └── testRaceOnLogFile2025-01-21-17.log
---
> │ ├── testRaceOnLogFile2025-01-21-17.log
> │ └── testRaceOnLogFile2025-01-21-18.log
25487c25489
< 4748 directories, 20736 files
---
> 4748 directories, 20738 files
63 smoke tests app are built in both cases.
Smoke test for a legacy renamed service "apigateway" also builds and runs fine on my dev machine:
~/workspace/github/aws-sdk-cpp]
my-super-duper-dev-machine $ ./buildRefactor/generated/smoke-tests/apigateway/apigateway-smoke-tests
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from APIGatewaySmokeTestSuite
[ RUN ] APIGatewaySmokeTestSuite.GetDomainNamesSuccess
[ OK ] APIGatewaySmokeTestSuite.GetDomainNamesSuccess (61 ms)
[----------] 1 test from APIGatewaySmokeTestSuite (61 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (96 ms total)
[ PASSED ] 1 test.
-DBUILD_ONLY logic was also tested.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for verifying.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changes look fine overall. Just a minor comment and request to verify if name change affects smoke tests.
# This is the 1st commit message: s3 code gen updates # This is the commit message #2: fixes # This is the commit message #3: updates # This is the commit message #4: updates # This is the commit message #5: compiling version # This is the commit message #6: updates # This is the commit message #7: test changes # This is the commit message #8: updates # This is the commit message #9: temp changes in s3client # This is the commit message #10: for debug # This is the commit message #11: Update API model # This is the commit message #12: corrects the dual-stack endpoint configuration for cognitoidp Added DeleteContactFlowVersion API and the CAMPAIGN flow type AWS IoT SiteWise now supports ingestion and querying of Null (all data types) and NaN (double type) values of bad or uncertain data quality. New partial error handling prevents data loss during ingestion. Enabled by default for new customers; existing customers can opt-in. Documentation-only update to address doc errors Documentation-only update: clarified the description of the shareDecaySeconds parameter of the FairsharePolicy data type, clarified the description of the priority parameter of the JobQueueDetail data type. Added `DigitGroupingStyle` in ThousandsSeparator to allow grouping by `LAKH`( Indian Grouping system ) currency. Support LAKH and `CRORE` currency types in Column Formatting. This release adds support for the topic attribute FifoThroughputScope for SNS FIFO topics. For details, see the documentation history in the Amazon Simple Notification Service Developer Guide. Increasing entryPoint in SparkSubmit to accept longer script paths. New limit is 4kb. # This is the commit message #13: upgrade smithy version (#3263) # This is the commit message #14: Refactor and simplify logic in sdksCommon cmake script to not rely on c2j model (#3251) # This is the commit message #15: Update API model # This is the commit message #16: Adds multi-turn input support for an Agent node in an Amazon Bedrock Flow Docs Update for timeout changes Rename WorkSpaces Web to WorkSpaces Secure Browser AWS Elemental MediaLive adds a new feature, ID3 segment tagging, in CMAF Ingest output groups. It allows customers to insert ID3 tags into every output segment, controlled by a newly added channel schedule action Id3SegmentTagging. # This is the commit message #17: Fix serialization for query xml # This is the commit message #18: Update API model # This is the commit message #19: Added "future" allocation type for future dated capacity reservation # This is the commit message #20: add stable order for smoke tests + restoring client codegen (#3265) # This is the commit message #21: add protocol test models (clients+tests) to the repo
Issue #, if available:
cmake logic depends on c2j file list in
tools/code-generation/api-descriptions
This does not make much sense anymore since
generated/src
source treei.e. cmake can rely solely on the source tree
tested by building both full main and this branch and by comparing the cmake output and resulting build/install directories, i.e.
Description of changes:
Check all that applies:
Check which platforms you have built SDK on to verify the correctness of this PR.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.